1. Field of the Invention
The present invention relates to the field of imaging, the field of computer assisted imaging, the field of digital imaging, and the field of automatically controlled enhancement of specific attributes of digital imaging data such as contrast.
2. Background of the Art
Except perhaps in the case of artistic effects, it is desirable that images, including digital images, reveal all the detail in a scene without creating an unnatural look. Faithful replication is an important goal for the capability of any imaging system. Contrast enhancement has, therefore, been an important objective in the image processing art. Automatic contrast adjustment has been a particularly sought after, but elusive objective. The concept of contrast in imaging technology is a broad term, especially when discussed in more than purely technical terms. Fundamentally, contrast concerns the visibility of detail in the image, or the ability of one detail or difference in an image to be visually differentiation from another detail. One view of contrast is in terms of the increment of brightness that can be discerned against a background of given brightness. This phenomenon can be seen in both a local and a global sense.
The edge of an object in an image is a local feature. When the difference in brightness across such an edge is increased, the local contrast is raised and the image is usually perceived as sharper. There is a whole spectrum of algorithms in image processing that concerns local contrast enhancement, ranging from sharpening filters such as those discussed in John. C. Russ, “The Image Processing Handbook”, 2nd Edition, CRC Press, Boca Raton, Fla., 1995 to algorithms that manipulate brightness within regions of images such as those described in S. M. Pizer, J. D. Austin, R. Cromartie, A. Geselowitz, B. ter Haar Romeny, J. B. Zimmerman and K. Zuiderveld, “Algorithms for adaptive histogram equalization”, Proc. SPIE, 671, 132 (1986). Some more recent examples of the latter include J. A. Stark and W. J. Fitzgerald, “An alternative algorithm for adaptive histogram equalization”, Graphical Models and Image Processing, 58, 180 (1996) or U.S. Pat. No. 6,148,103 concerning a method for improving contrast in picture sequences. U.S. Pat. No. 5,581,370 discloses a method of improving the contrast of a natural scene that makes use of local image histograms and U.S. Pat. No. 5,426,517 describes automated tone correction using filtered histogram equalization. In Y.-T. Kim, “Mean based bi-histogram equalization: a novel extension of histogram equalization preserving brightness”, Proc. IASTED International Conf. Signal and Image Processing (SIP '96), 310 (1996), the importance of keeping the mean brightness of the image unchanged during local manipulation of contrast is emphasized. The results of manipulating local contrast depend on the specific spatial pattern of brightness in the image.
The global view of image contrast is based on the observation that, in very dark or very light image areas, the human eye can discern only relatively large brightness differences, while smaller differences can be distinguished at intermediate brightness levels. This is discussed, for instance, in Chapter 2 of W. K. Pratt, “Digital Image Processing”, Vol. 1, Wiley, New York (1978). As a result, a change in the distribution of brightness in an image can hide or reveal detail. It is well know to apply such changes manually using a power law transformation such as that found, for instance, in the “Gamma Correction” function of Paint Shop Pro 7 (Jasc Software, Inc., 7095 Fuller Road, Eden Prairie, Minn., 55344). The distribution of brightness is characterized by a histogram, which is a graph that represents the frequency of occurrence of specific brightness levels within an image. Note that because the graph is a frequency distribution rather than a positional distribution, if the location of image pixels is randomly permuted in the image, the histogram does not change. Thus, global contrast manipulations that modify the distribution of brightness are not influenced by the specific spatial pattern of brightness in the image. This has led to difficulties in applying optimal global contrast adjustment to images in an automated fashion. The present invention is concerned with such global adjustment.
Among the methods for modifying the global contrast of images are those based on histogram equalization. Histogram equalization maintains the brightness ranking of image pixels but redistributes the brightness values so that an equal number of pixels have each possible brightness value. This, however, can lead to a nighttime scene being rendered or converted to a daytime scene. Other methods of histogram adjustment described in R. Humel, “Image enhancement by histogram transformation”, Comput. Graphics Image Processing, 6, 184 (1977) and W. Frei, “Image enhancement by histogram hyperbolization”, Comput. Graphics Image Processing, 6, 286 (1977). A. Mokrane in “A New Image Contrast Enhancement Technique Based on a Contrast Discrimination Model”, Graphical Models and Image Processing, 54, 171 (1992) discusses a brightness transformation based on a mathematical model that follows a power law. U.S. Pat. No. 5,265,200 discloses a method of fitting weighted second order functions to a gray scale level histogram by linear regression and using one of the fitted functions to modify the image. Another approach to global contrast manipulation involves specifying the desired form of the histogram. Examples of such approaches include V. F. Nesteruk, “Optimum nonlinear contrast statistical image converter and an evaluation of its efficiency”, Soviet J. Optical Technology, 48, 647 (1981) and V. F. Nesteruk and N. N. Porfiryeva, “Contrast law of light perception”, Optics and Spectroscopy, 29, 606 (1970). None of these global correction methods, however, give fully satisfactory results, especially with regard to reliably acceptable enhancement.
Additional art of relevance to global histogram adjustment includes the following. Japanese Pat. 9-149277 (Application No. JP 95301703) describes a method of adjusting the lightness of an image on the basis of the histogram maximum. Japanese Pat. 5-176220 (Application No. JP 91343518) concerns an automatic exposure control for a camera using fuzzy reasoning, which estimates mean image brightness and controls exposure based on the number of pixels having a brightness lower than the mean. U.S. Pat. No. 5,883,984 describes a hardware method of contrast adjustment using I in the HSI color space and the mean and median values of the RGB color channels. U.S. Pat. No. 5,396,300 discloses a contrast correction device for video signals using a circuit to determine if the image is bright or dark and a second circuit to apply a gamma correction. U.S. Pat. No. 5,926,562 teaches a method of exposure compensation by selecting a gamma correction on the basis of at least one of a reference minimum value, a reference maximum value and the average value of the brightness of an image. U.S. Pat. No. 5,812,286 discloses a method of color and contrast correction based on the minimum, median and maximum values of each color channel, along with a user-supplied parameter. U.S. Pat. No. 5,414,538 describes a method of contrast correction in which bounds of a brightness histogram are compared to thresholds and, if the thresholds are exceeded, the bounds and thresholds are used to form a gamma correction for the brightness. U.S. Pat. No. 5,712,930 teaches the selection of a gamma correction function from among several such functions by means of a neural network. U.S. Pat. No. 4,731,671 discusses a method where image contrast is automatically determined as a function of the standard deviation of a sample of tone values that is selected from a plurality of samples corresponding to a plurality of contrast intervals based on the shape of the histogram. In U.S. Pat. No. 4,654,722 is described a related procedure based normalizing a histogram of a sample of tone values selected from an image dependent “floating” contrast interval. U.S. Pat. No. 5,937,090 discloses a image enhancement method using quantized histogram equalization, which retains the mean input brightness as the mean output brightness. Though this is a local correction method, the disclosure also contemplates adding an offset to very low mean brightness values and subtracting an offset from very high mean brightness values. U.S. Pat. No. 5,450,502 teaches forming a global histogram of intensity and operating on it with a filter that flattens peaks and valleys, though the claims also require calculation of local histograms. U.S. Pat. No. 5,347,374 describes cascaded histogram processing, wherein a histogram is modified with a tone reproduction curve and smoothed in a first image processing module and then directed to a second processing module in which a second tone reproduction curve further modifies the histogram. Finally, the following paper describes lightness modification with a sigmoidal transfer curve: G. J. Braun and M. D. Fairchild, “Image lightness rescaling using sigmoidal contrast enhancement functions”, J. Electronic Imaging, 8, 380 (1999). No mathematical form for any sigmoidal function is given and it is most probable that the transformation was effected by means of an interpolation (such as a spline) through a series of hand-crafted, and therefore not dynamically adjustable, points. Further, this paper is concerned with mapping from the color gamut of one device to another rather than with image enhancement.